Caching is computer technology that allows computer processes to be accelerated. Generally, a computer uses multiple levels of memory types of varying sizes and speeds, with cheaper memory being generally slower and bigger in storage size and more expensive memory being generally faster and smaller in storage size. As faster memory is generally small in storage size, only a limited amount of data can be stored on the faster memory types in the computer. Generally described, caching attempts to anticipate what data items are needed by the computer in the future and attempts to keep those data items in the limited amounts of the faster memory types in order to improve performance by reducing access times to the data items. These data items can be maintained in a cache data structure in the computer memory.
Generally described, a cache (e.g., an application cache) includes a data structure that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache may include values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (e.g., cache hit), the request can be served by reading the cache instead of calculating the data or retrieving the data from elsewhere. If reading from the cache is faster than calculating or retrieving the data, then the request is served faster. Otherwise (e.g., cache miss), the data has to be recomputed or fetched from its original storage location, which can be comparatively slower. Generally, the greater the number of requests that can be served from the cache, the faster the overall system performance.
Generally, cache sizes are small relative to bulk computer storage. Nevertheless, caches have proven themselves useful in many areas of computing because access patterns in typical computer applications have locality of reference. For example, references exhibit temporal locality if data is requested again that has been recently requested already. In another example, references exhibit spatial locality if data is requested that is physically stored close to data that has been requested already. Thus, caches can be beneficial, despite being able to fit only a portion of data stored in the bulk computer storage.